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MFTHODS, APPARATUS AND DATA STRUCTURES FOR 
PRESERVING ADDRESS AND SERVICE LEVEL INFORMATION IN A 

VIRTUAL PRIVATE NETWORK 



TECHNICAL FIELD 

The present invention concerns methods, apparatus and data structures 
for aggregating traffic, \Mch may orig^te from various media transport types, 
for presentation to a router, such as an access router of a network. Further, the. 
10 traffic aggregation performed by the present invention maybe done suchiJmt 
customers can be identified and such that customer device addressing 
information is available. Moreover, the traffic aggregation performed by the 
present invention may be done such that the service provided to a group of 
customers may be monitored; multicast groups are secure; and the access router 
can control access to services, facilitate virtual private networks, and facilitate the 
provision of different quality of service and/or class of service levels. 
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BACKGROUND 

The description of art in this section is not, and should not be interpreted 
to be, an admission that such art is prior art to the present invention. 

Although networking software and network reference models are known to 
those skilled in the art, they are introduced here for the reader's convenience. To 
reduce their complexity, networks maybe organized as a series of layers, each one 
built upon the one below it as shown in Figure 1. Each layer functions to offer 
certain services to the higher layer, thereby shielding those higher layers from the 
details of how the offered services are actually implemented. The entities 
comprising the corresponding layers on different machines are caUed "peers". 
Such peers use rules and conventions, also referred to as the layer n protocol, to 
communicate with each other as depicted by the dashed lines in Figure 1. 
Actually, no data are directly transferred from layer n on one machine to layer n 
on another machine. Rather, in the machme transmitting the data, each layer 
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passes data and control information to the layer immediately below it, until the 
lowest layer Qayer i) is reached. Below layer i, is a physical mediimi no through 
which actual communications take place. At the machine receiving the data, each 
layer passes data and control information to the layer immediately above it until 
5 the highest layer is reached. Thus, referring to Figure i, actual commimications 
take place via the solid lines and the physical medium no, while virtual 
peer-to-peer communications occur via the dashed lines. 

Still referring to Figure i, interfaces are arranged between adjacent layers. 
Each of these interfaces defines primitive operations and services that the lower 
10 layer offers to the upper layer. 

The set of layers and protocols maybe referred to as a "network 
architecture". A list of protocols used by a system, one protocol per layer, maybe 
referred to as a "protocol stack" or "protocol suite". 

Figure 2 illustrates a comparison of the Open Systems Interconnection (or 
15 "OSI'O reference model 2io for network architectures and the transfer control 
protocol/Internet protocol (or "TCP/IP") reference model 220 for network 
architectures. Although those skilled in the art will be familiar with both 
reference models, each is introduced below for the reader's convenience. 

As shown in Figure 2, the OSI reference model 210 has seven (7) distinct 
20 . layers; namely, (i) a physical layer 211, (ii) a data link layer 212, (iii) a network 
layer 213, (iv) a transport layer 214, (v) a session layer 215, (vi) a presentation 
layer 216, and (vii) an application layer 217. Each layer is briefly introduced 
below. 

The physical layer 211 deals with transmitting raw bits over a 
25 commimications channel. Thus, the physical layer is typically concemed with 
mechanical, electrical, optical, and procedural interfaces, as well as the physical 
transmission mediirai (e.g., twisted copper pair, co-axial cable, optical fiber, etc.) 
that lies below the physical layer. 

The data link layer 212 functions to transform a raw communications 
30 facility into a line that appears free from imdetected transmission errors to the 
network layer 213. The data link layer 212 does this by having the sending host 
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segment its data into "data frames^ transmitting these frames to the recdving 
host, and processing "acknowledgement frames" sent back from the receiver. 

The network layer 213 functions to control the operation of a subnetwork 
between the hosts and controls the routins of packets between the hosts. 

The transport layer 214 functions to accept data from the session layer 2^ 
and segment this data into smaller units, if necessary, for use by the network 
layer 213. The transport layer 214 also determines a type of sendee (e.g., 
error-free, point-to-point) to provide to the session layer 215. Further, the 
transport layer 214 confrols the flow of data between hosts. The transport layer 
214 is a true "end-to-end" layer, from source host to destination host, since a 
program on the source machine converses with a similar program on the 
destination machine, using message headers and control messages. 

The session layer 215 functions to allow different machines to estabUsh 
sessions between them. The session layer 215 may manage dialog control and 
maintain synchronization The presentation layer 215 concerns the syntax and 
semantics of infonnation transmitted. The application layer 216 may function to 
define network virtual terminals that editors and other programs can use, and to 
transfer files. 

In recent decades, and in the past five (5) to ten (10) years m particular, 
computers have become interconnectedby networks by an ever increasing extent; 
initially, via local area networks (or "LANs"), and more recently via LANs, wide 
area networks (or WANs) and the Internet. In 1969, the Advanced Research 
Projects Agency (ARPA) of the U.S. Department of Defense (DoD) deployed 
ARPANET as a way to explore packet-switchmg technology and protocols that 
could be used for cooperative, distributed, computing. Early on, ARPANET was 
used by the TELNET application tiiat permitted a angle terminal to work with 
different types of computers, and by tiie file transfer protocol (or "FTP") v*ich 
permitted different types of computers to tiransfer files fixjm one another. In the 
early 1970s', electronic mail became tiie most popular application which used 
) ARPANET. 

This packet svdtching technology was so successful, that the ARPA appUed 
it to tactical radio communications (Packet Radio) and to satellite 
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communications (SATNET). However, since these networks operated in very 
different communications environments, certain parameters, such as maximum 
packet size for example, were different in each case. Thus, methods and protocols 
were developed for "internetworking" these different packet switched networks. 
5 This work lead to the transmission control protocol (or *TCP") and the intemet 
protocol (or "IP**) which became the TCP/IP protocol Siiite. Although the TCP/IP . 
protocol suite, wliich is the foimdation of the Intemet, is known to those skilled in 
the art, it is briefly described below for the reader's convenience. 

As shown in Figure 2, the TCP/IP reference model 220 includes a physical 
10 layer 221, a network access layer 222, an intemet layer 223, a transport layer 224, 
and an application layer 225. Each of these layers is briefly introduced below. 

The physical layer 221 defines the interface between a data transmission 
device (e.g., a computer) and a transmission medium (e.g., twisted pair copper 
wires, co-axial cable, optical fiber, etc.). It specifies the characteristics of the 
15 transmission mediiim, the nature of the signals, the data rate, etc. 

The network access layer 222 defines the interface between an end system 
and the network to which it is attached. It concerns access to, and routing data 
across, a network. Frame relay is an example of a network access layer. 

The intemet layer 223 functions to permit hosts to inject packets into any 
20 network and have them travel independenfly to the destination machine (which 
may be on a different network). Since these packets may travel independently, 
they may event arrive in an order other than the order in which they were sent. 
Higher layers can be used to reorder the packets. Thus, the main function of the 
. intemet layer 320 is to deliver (e.g., route) IP packets to their destination. 

25 The transport layer 224 is an end-to-end protocol. For example, the 

transmission control protocol (or "TCP") is a reliable connection-oriented 
protocol that allows a byte stream originating on one machine to be delivered, 
without error, on any other machine on the Internet. More specifically, the TCP 
protocol fragments an incoming data stream into discrete messages, each of 

30 which is passed to the intemet layer 223. At the destination, the TCP protocol 
reassembles the received messages into an output stream. 
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The TCP /IP model 220 does not have session and presentation layers. 
Instead, an appUcation layer 225 contains an of the higher-level protocols that are 
used to support various types of end use appHcations (e.g., the simple mail ; 
transfer protocol (or "SMTP") for e-mail, the file transfer protocol (or "FTP"), 
etc.)- 

The TCP /IP model does not define what occurs below the internet layer 
223, other than to note that the host has to connect to the network using some 
protocol so that it can send IP packets over it. This protocol varies from host to 
host and network to network. 

Basically, each of the layers encapsulates, or converts, data in a higher 
layer. For example, referring to Figure 4. user data 400 as a byte stream is 
provided with a TCP header 402 to form a TCP segment 410. The TCP segment 
410 is provided with an IP header 412 to form an IP datagram 420. The IP 
datagram 420 is provided with a network header 422 to define a network-level 
packet 430. The network-level packet 430 is then converted to radio, electrical, 
optical (or other) signals sent over the transmission medium at a specified rate 
with a specified type of modtilation. 

The TCP header 402, as illustrated in Figure 5, includes at least twenty 
(20) octets (i.e., 160 bits). Fields 502 and 504 identify ports at the source and 
destination systems, respectively, that are using the connection. Values in the 
sequence number 506, acknowledgement number 508 and window 516 files are 
used to provide flow and error control. The value in the checksum field 518 is 
used to detect errors in the TCP segment 410- 

Figures 6A and 6B illustrate two (2) alternative IP headers 412 and 412', 
respectively. Basically, Figure 6Adei»cts the IP protocol (V^eraon 4) that has 
been used. Figure 6B depicts a next generation IP protocol (Version 6) that, 
among other thmgs, provides for more source and destination addresses. 

More specifically, referring to Figure 6A, the four (4) bit version field 602 
indicates the version number of the IP, m this case, version 4. The 4-bit Internet 
header length field 604 identifies the length of the header 412 in 32-bit words. 
The 8-bit type of service field 606 mdicates the service level thatthe IP datagram 
420 should be given. The 16-bit total length field 608 identifies the total length of 
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the IP datagram 420 in octets. The 16-bit identification field 610 is tised to help 
reassemble fragmented user data carried in multiple packets. The 3-bit flags field 
612 is used to control firagmentation. The 13-bit firagment offset fidd 614 is used 
to reassemble a datagram 420 that has become firagmented. The 8-bit time to live 
5 field 616 defines a maximum time that the datagram is allowed to exist within the 
network it travels oven The 8-bit protocol field 618 defines the higher-level 
protocol to which the data portion of the datagram 420 belongs. The 16-bit 
header checksum field 620 permits the integrity of the IP header 412 to be 
checked. The 32-bit source address field 322 contains the IP address of the 
10 sender of the IP datagram 420 and the 32-bit destination address field contains 
the IP address of the host to which the IP datagram 120 is being sent. Options 
and padding 626 maybe used to describe special packet processing and/or to 
ensure that the header 412 is a complete multiple of 32-bit words. 

Referring to Figure 6B, the four (4) bit version field 602 indicates the 
15 version number of the IP, in this case, version 6. The 4-bit priority field 628 

enables a sender to prioritize packets sent by it. The 24-bit flow label field 630 is 
used by a source to label packets for which special handling is requested. The 
16-bit payload length field 632 identifies the size of data carried in the packet. 
The 8-bit next header field 634 is used to indicate whether another header is 
20 present and if so, to identify it. The 8-bit hop limit field 636 serves to discard the 
IP datagram 420 if a hop limit (e.g., the niraiber of times the packet is routed) is 
exceeded. Also provided are 128-bit source and destination address fields 322' 
and 324', respectively. 

Having described the TCP/IP protocol stack 220, the routing of a TCP/IP 
25 packet is now described. A TCP/IP packet is communicated over the Internet (or 
any internet or intranet) via routers. Basically, routers in the Internet use 
destination address information (Recall fields 624 and 624'.) to forward packets 
towards their destination. Routers interconnect different networks. More 
specifically, routers accept incoming packets from various connected networks, 
30 use a look-up table to determine a network upon which the packet should be 
placed, and routes the packet to the determined network. 
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Rgure 7, wbich includes Kgures 7 A throvigh 7C, illustrates the 
communication of data from a sender, to a receiver, using the TCP/IP protocol 
stack. Referring first to Rgure 7A, an application protocol 702 prepares a block 
of data (e.g., an e-mail message (SMTP), a file (FTP), user input (TELNET), etc.) 
5 400 for transmission. Before the data 400 are sent, the sending and receiving 
applications agree on a format and encoding and agree to exchange data (Recall, 
e.g., the peer-to-peer communications depicted with dashed lines in Figure 1,). If 
necessary, the data are converted (character code, compression, encryption, etc.) 
to a form expected by the destination device. 
10 The TCP layer 704 may segment the data block 400, keeping track of the 

sequence of segments. Each TCP segment 410 includes a header 402 containing a 
sequence number (recall field 506) and a frame check sequence to detect errors. 
A copy of each TCP segment is made so that if a segment is lost or damaged, it can 
be retransmitted. "When an acknowledgement of safe receipt is received from the 
15 receiver, the copy of the segment is erased. 

The IP layer 706 may break the TCP segment into a number of datagrams 
420 to meet size requirements of networks over which the data will be 
communicated. Each datagram includes the IP header 412. 

A network layer 708, such as frame relay for example, may apply a header 
20 and trailer 422 to frame the datagram 420. The header may include a connection 
identifier and the trailer may contain a frame check sequence for example. Each 
frame 430 is then transmitted, by the phyacal layer 710, over the fransmission 
medium as a sequence of bits. 

Figure 7B illustrates the operation of the TCP/IP protocol stack at a router 
25 m the network. The physical layer 712 receives the incoming signal 43© from the 
fransmission medium and interprets it as a frame of bits. The network (e.g., 
frame relay) layer 714 then removes the header and frailer 422 and processes 
them. A frame check sequence may be used for error detection. A comiection 
number may be used to identify the source. The network layer 714 then passes 
30 the IP datagram 420 to the IP layer 718. 

The IP layer examines the IP header 412 and makes a routing dedsion 
(Recall the destination address 324, 324'). A local line confrol (or "LLC") layer 
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720 uses a simple network management protocol (or "SNMP**) and adds a header 
750 that contains a sequence number and address information. Another network 
layer 722 (e.g., media access control (or "MAC")) adds a header and traUer 760. 
The header may contain address information and the trailer may contain a frame 
5 check sequence. The physical layer 724 then transmits the frame 450 over 
another transmission medium. 

Figure 7C illustrates the operation of the TCP/IP protocol stack at a 
receiver. The physical layer 732 receives the signals from the transmission 
medium and interprets them as a frame of bits. The network layer 734 removes 

10 :• the header and trailer 760 and processes them. For example, the frame check 
sequence in the trailer may be used for error detection- The resulting packet 440 
is passed to the transport layer 736, which processes the header 750 for flow and 
error control. The resulting IP datagram 420 is passed to the IP layer 738, which 
removes the header 412. Frame check sequence and other control information 

15 may be processed at this point. 

The TCP segment 410 is then passed to the TCP layer 740, which removes 
the header 402 and may check the frame check sequence. (In the event of a 
match, the match is acknowledged and in the event of a mismatch, the packet is 
discarded.) The TCP layer 740 then passes the data 400 to the application layer 
20 742. If the user data was segmented (or fragmented), the TCP layer 740 
reassembles it. Finally, the application layer 742 performs any necessary 
transformations, such as decompression and decryption for example, and directs 
the data to an appropriate area of the receiver, for use by the receiving 
application. 

25 The present inventors believe that most of the world's networks are, or will 

be, based on the Internet Protocol (or **IP'*). There are at least three (3) 
assumptions imderlying this belief. First, IP separates applications (or services) 
from transport (e.g., data link technology). The present inventors believe that 
value added services will be IP-based, due in part to favorable price-performance 

30 • curves of IP access technology and the way in wiiich IP can inter-operate with 
other technologies. Second,IPqualityof service (or "QoS^O is emerging. These 
QoS mechamsms can be applied to the spedfic applications and sendees (e.g.. 
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audio-visual multicast, conferencing, high speed access such as via DSL. IP 
derivedlines, IP tdephony, IP fax. IP Centrex, Internet service provider (or 
"ISPl services such as e-mail, Internet access, authorization, authentication and 
accounting,andbiffing,andun3fiedmessa^g)ofindividualcustomers. Various 
types of applications may demand various levels of qualityof service. For 
example, a voice over Internet appUcation may require low delays, but may 
tolerate some packets being dropped, to the extent that such dropped packets 
cannot be perceived or are not annoying to users. This is because it would be 
pointless to retransmit erroneous packets in such a real-time application. Data 
transportmaytoleratedelaysbutwillnottoleratetransmissionerrors. Video . 

over the Internet will require high bandwidth but may tolerate some dropped 
packets (again, to the extent that such dropped packets would not be perceived 
by orbeamxoyingto,acustomer). Third, data competitive (or certified) local 
exchange carriers (or "DLECs") - that is, companies that provide high speed 
access to the Internet - currently provide integrated IP services using 
asynchronous transfer mode (or "ATM") transport. The present inventors beheve 
that as lower cost link layer technologies are deployed, such as ^gabit Ethernet,, 
for example, DLECs wiU abandon ATM. 

With this background in mind, the present inventors propose a 
,o multi-servicelocalaccessandtransportarea(or«LATA»)IPnetworkwiththe 
following two (2) design goals in mind. First, it should be simple for existing and 

potential customers to use the proposed LATA IP network. Second, the LATA IP 
networkshouldberobustandfle3dble,whilehavingalowoperatingcost. The 

present inventors believe that customer simplicity can be achieved by (i) 
25 elhninatingorminimiringchangestoexistinglayeriandscustomerinterfaces 

(so that existing customers maybe retained) and (ii) providing new, low cost, 
high value IP interfaces to customers (such as Fast Ethernet and Gigabit 
Ethernet). The present inventors further believe that the LATAIP network canbe 
robust, flexible, and have low operatii« costs by CO minimizing complexity (by 
30 isolating subsystems with different component technologies and separating 

application functionality from the underlying transport network), (ii) minimizmg 
operations, (iii) providing the abiUty to route traffic for services vMch have 
different topology and volume assumptions, and (iv) ensuring reUability by usmg 
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off-the-shelf components and standard protocols (thereby eliminating 
customization) and by providing redundant equipment and facilities. 

The LATA IP network envisioned by the present inventors may use 
off-the-shelf routers. These routers may function to (i) provide access to 
5 customers, (ii) interconnect networks, and/or (iii) provide routing between 
, intranetwork elements. Thus, the LATA IP network may use three (3) different 
types of routers. In the LATA IP network, access routers may be distributed 
towards the edge of the network and may provide individual customer IP 
interfaces into the network. Thus, the access router may act as a universal IP 
10 edge device for diverse customer access methods. Interconnection routers may 
be centralized with the IP LATA and may provide a small niraiber of (e.g., high 
bandwidth) external interfaces to the other carrier's (or enterprise customer's) 
) network(s). Finally, routers maybe deployed, as needed, throughout the IP LATA 
to consolidate traffic and to minimize the cost of traffic transport between 
15 elements of the IP LATA. 

One aspect of the present invention concems the challenge of aggregating 
a nimiber of physical connections firom a number of potentially diverse 
customers, for connection to an edge router. For example, standards-based 
routers that can handle 128 Gbps bandwidth are currently available. However, 

20 such routers cannot accommodate the physical connections of the tens or 
hundreds of thousands of individual services that they could otherwise . 
accommodate. For example, assuming that customers had a very high end 10 or 
100 Mbps service (or communications access links capable of such service levels), 
such routers could process the data flow from 12,800 or 1,280 customers, 

25 respectively, but could not accommodate those numbers of physical connections. 
Naturally, a larger number of physical connections (e.g., for lower end service(s)) 
coiddnotbe accommodated. 

Digital subscriber line access multiplexers (or "DSLAMs'O maybe used to 
concentrate traffic in asynchronous digital subscriber line (or "ADSL") 
30 implementations by using time diviaon multiplexing. Basically, a DSLAM can 
accept twisted copper pairs supporting ADSL service and provide them on virtual 
channels on a shared common commxmications medium, such as an OC3 (e.g.. 
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«5.5a Mbps) fiber chaimel. However, an asynchronous transfer mode (or 
™J.chisneeded,osv.itcha>esephysiedeonnecaonstovirtml« 
therebynecessitatinganAmswitchportforeachcustomerconnection.A«^ 
Lplysicanyrequiringalot of space, usingaDSLAMforttuspur^e^onldbe 

expensiveonaperportbasU.Ttos,taproveatechmquesareneededto 
^^te physfcal connecttons, for exan.ple, for presenution to an access router. 

Another aspect of the present invention concemsthe challengeof . 
separating customer services from customer access technolopes (e.g., ML, 
plmeMay.Gigabyte Ethernet), in this way.avariety of services couldbe 

provided toavariety of potential customerswithoutregardfor the waymvAuch 
such potential customers access the IP LATA netv«>rk. 

SUMMARY 

The present invention may provide an aggregation unit to aggregate 
; physical connections from customers for presentation to an access router and to 
a.igr^ate.rafacfromasharedlinlc(s)ft.m.heaccessrouter. ThesetocUons 
n^y^accomplishedbyconfiguringlogicalportsoftheaggregadonumt^ach 

J each has a unique layer a (e.g., MAO address or some other umque b>t strmg 
(alsoreferr^toas-contextinfonnatiomassociatedvdthit. Suchcontext 
0 Lnna.ionmayreplace,atleasttosomeextent,layer.(e.g.,address,heaa^ 
iirformation on packetsacceptedbythelogical port, in one embodiment, the 
context information mayindude customer-specific information, mfonnanon 
locating thelogicalport^ithtothenetwork, and/or dass of servicemformauon. 

•ms context information, which depends solely on the logical port, can be 
„ extemledtoindudequalityofser™ceinformaUon.Sucbqualityofserv,ce 

iuf ormation may convey network requirements inherent in the apphcatton 
„hidtaninboundpad«t(s)isassodated.aadmaybedenvedfromlayer3and 

h,yer 4 information in the inbound padtetCs). Thus context informatron may 
indudeapaAet-independentpartassodatedwithalo^portanda 

30 padcet^ependantpartdetemunedftomaninboundpad^s). n^^-^' 
!tring-or"contextinfotma.ion"isnotintendedtobelimitedtocmbguo,«b.ts. 

aod is to indude non.<onliguous bits as can be appredated ftom 36. 
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If it can be assumed that IP addresses are globally unique, the layer 2 (e.g., 
MAC) address of the customer device connected with the port can be associated 
with, and therefore determined from, the IP address of the attached device. 
Otherwise (or in addition), the layer 2 (e.g., MAC) address of the customer device 

5 connected with the port can be determined using some type of address resolution 
technique (e.g., resolving the address with a protocol, such as ARP for example, 
typically by broadcasting a request for an address), and/or snooping (e.g., 
examining the layer 2 source address of an inboimd (ingress) packet). Thus, for 
example, if the IP addresses are dynamically assigned to customer devices, then 

10 the aggregation unit may periodically poll (e.g., via an address resolution protocol 
or "ARP"* broadcast) the attached device(s) for its layer 2 (e»g., MAC) address, 
and/ or may examine the layer 2 source address of inboxmd packets. 

When a packet is received from a customer, layer 2 header information 
(e.g., the source and destination layer 2 (e.g., MAC) addresses) maybe removed 

15 and a imique bit string (or "context information"), a part of which is associated 
, with a logical port or interface (which is associated with the physical port), and a 
part of which is based on layer 3 and/or 4 information in the packet, may be 
added. Preferably, these operations will not alter the "footprint" of the packet. 
To reiterate, these bits that replace layer 2 header information (e.g-, the source 

20 and destination layer 2 (e.g., MAC) addresses), may be referred to as "context 

information". Again, context information may include a packet-independent part 
associated with a logical port and a packet-dependant part determined from an 
inboxmd packet(s). Traffic received at the logical ports is then aggregated onto a 
high bandwidth physical link(s) to the access router. 

25 When a packet is received from the access router, the aggregation \mit 

forwards it to the logical port associated with at least some bits of the bit string 
(i.e., of the context information) that reside in the place of the layer 2 (address) 
header. The destination layer 2 (e.g., MAC) address (or the other bits in the place 
of the layer 2 address) is then replaced with the layer 2 (e.g., MAC) address of the 

30 customer device associated with the port. To reiterate, the layer 2 (e.g., MAC) 
address of the customer device may be derived from the layer 3 destination 
address (if it can be assumed that layer 3 addresses are globally unique), or. 
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alternatively may have been determined using an address resolution technique, 
and/or snooping. 

The preset invention may also support multicast groups by checking at 
least a part of the unique bit string (i.e., context information) which had been ; 
5 inserted in the layer 2 header space to determine vdiether or not the customer 
associated with that port is permitted to jom the multicast group. The present 
invention may monitor the service provided to a group of customers, that group 
of customers being defined by at least a portion of the unique bit string (i.e., 
context information) which had been inserted in the layer 2 header space. 

10 The present invention may also function serve to limit or control access to 

various services thereby performing a firewall function. In this regard, an access 
router may permit or deny a packet based on at least a portion of the \mique bit 
string (i.e., context information) which had been inserted in the layer 2 header 
space. The present invention may further function to facilitate the proviaon of 

15 different quality of service levels. A particular quality of service maybe mdicated 
by at least a part of the unique bit string (i.e., context information) which had 
been inserted in the layer 2 header space. 

The present invention may also function to enable -wrtual private networks 
since it preserves layer 2 header information or a unique bit string (or context 

20 information) which had been inserted in tiie layer 2 header space. 



BRIEF DESCRIPTION OF THE DRAWINGS 

Figture 1 illustrates the way in which network commimications schemes 
maybe described by a stack of protocols. 

Figure 2 compares the OSI reference model and the TCP/IP protocol suite. 

Figure 3 illustrates internet protocol (or "IP") global addressmg. 

Figure 4 illustrates the manner in which data is encapsulated by a TCP 
header, an IP header, and a network header in accordance with the TCP/IP 
protocol suite. 

Figure 5 illustrates the fields of a TCP header. 
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Figiires 6A and 6B illustrate the fields of Version 4 and Version 6, 
respectively, of the IP header. 

Figures 7A through 7C illustrate the transmission of data over a network in 
accordance with the TCP/IP protocol suite. 

5 . Figure 8 is a high level diagram of a network that the present invention 

may be used to access. 

Figure 9 is an example of the network of Figure 8 in which services and 
applications are shown separated from transport. 

Figure 10 is a high level diagram of processes that may be performed by 
10 various aspects of the present invention. 

Figure 11 illustrates how various access technologies may interface with an 
access router of the network of Figure 8 or 9* 

Figure 12 illustrates fields of an Ethernet frame. 

Figure 13 illustrates an exemplary data structure specification of a unique 
15 bit string (or context information) that may be used in the present invention and 
that maybe administered in accordance with a network-wide plan. 

Figure 14 is a high-level block diagram of an exemplary aggregation imit. 

Figiore 15 illustrates a physical implementation of an exemplary 
aggregation unit. 

20 Figure 16 illustrates an exemplary implementation of management cards 

in the exemplary aggregation tinit of Figure 15. 

Figure 17 illustrates an exemplary implementation of customer facing 
interfaces (or ports) in the exemplary aggregation unit of Figure 15. 

Figure 18 illustrates an exemplary implementation of network facing 
25 interfaces in the exemplary aggregation unit of Figure 15 . 

Rgure 19 is a high level flow diagram which illustrates oi)erations which 
may be performed as a packet enters an IP network via an aggregation device and 
an (ingress) access router, and as a packet leaves an IP network via an (egress) 
:. access router and an aggregation device. 
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Figure 20 is a flow diagram of an exemplary method that maybe used to 
effect a logical port configuration function 

Figure 21 is a flow diagram of an exemplary method that maybe used to 
rffect a logical port aggregation function. 

Figure 22 is a flow diagram of an exemplary method that may be used to 
effect a link de-aggregation function. 

Figure 23 is a flow diagram of an exemplary method that may be used to. 
effect a multicast group monitoring function. 

Figure 24 is a flow diagram of an exemplary method that may be used to 
effect a customer group monitoring function. 

Figure 25 illustrates an exemplary data structtjre of access control 
information that maybe used by an exemplary access router. 

Figure 26 is a flow diagram of an exemplary method that may be used to. 
effect an access control function. 

Figures 27A and 27B are flow diagrams of exemplary methods that may be 
used to effect a virtiial private network addressing function as a packet enters tiie 
network (ingress) and as a packet leaves tiie network (egress), respectively. 

Figure 28 is a flow diagram of an exemplary method that maybe used to 
enable various service levels. 

Figure 29 illustrates an exemplary table that may be used by an exemplary 
aggregation device, to configure logical ports. 

Figure 30 illusti^tes an exemplary table tiiat may be used by an exemplary 
aggregation device, to convert a port layer 2 address (or information in tiie place 
of the layer 2 address) to a customs device layer 2 address. 

Figure 31 illustrates an exemplary table tiiat maybe used by an exemplary 
aggregation device, to associate multicast networks or subnetworks witii a virtual 
private network. 

Figure 32 illustirates an exemplary table that may be used by an exemplary 
access router, to control access to a network or to a network location. 
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Figure 33 illustrates an exemplary table, which may be used by an 
, exemplary access router, to encapsulate a packet so that layer 2 address 
information (or information in the place of the layer 2 address header) maybe 
preserved. 

5 Figure 34 illustrates an exemplary table, which may be used by an 

exemplary access router, to determine a layer 2 (e.g., MAC) address of a customer 
device based on a layer 3 address and/or bits in the place of information (e.g., 
address information) in a layer 2 header. 

Figure 35 illustrates an exemplary packet which may be sent by a customer 
10 and received by an aggregation unit. 

Figure 36 illustrates the modification, by an exemplary aggregation unit, of 
a packet sent from a customer and boimd for a network. 

Figure 37 illustrates the modification, by an exemplary access router, of a 
padcet sent from a customer, as forwarded by an aggregation imit, and boimd for 
15 • a network. 



DETAILED DESCRIPTION 

The present invention involves novel methods, apparatus and data 
structures for permitting customers to access a network, such as an IP network, 

20 and to help provide certain services. The following description is presented to 
enable one skilled in the art to make and use the invention, and is provided in the 
context of particular applications and their requirements. Various modifications 
to the disclosed embodiments will be apparent to those skilled in the art, and the 
general principles set forth below maybe applied to other embodiments and 

25 applications. Thus, the present invention is not intended to be limited to the 
embodiments shown and the inventors regard their invention as the following 
disclosed methods, apparatus and data structures and any other patentable 
subject matter. 

In the following, an exemplary environment in which the invention may 
30 operate is described. Then, fimctions that maybe performed by the present 
invention are introduced. Thereafter, processes, structuures, methods and data 



mmmmmmmmmmm 



m 



wo 02/190^^^ 

17 



/(fl^ PCT/US01/2473i 



10 



15 



20 



25 



stmcturesthatmaybeusedtoeffertthosefimctionsaredesa^^ lliereafter, 
the end-to-end processing of a packet in a system including exemplary 
aggregation units and access routers is described. RnaUy, some conclusions 
regarding various aspects of the present invention are provided. 

Figure 8 is a high level diagram of an environment 800 in which the 
present invention may operate. This environment 800 may include a LATA IP 
network 810, additional networks 820 such as an enterprise network, a portal 
Internet sendee provider (or "ISP") network, a peer ISP network, and an existing 
layer 2 service provider network. The networks 820 maybe interconnected with 
tiie LATA IP network 810 via interconnection router(s) 816. Customers 830, such 
as homes and businesses, maybe connected vAth the LATA IP network 810 via 
"access routers" 812. FinaUy, routers 814 may be provided within tiie LATA IP 
network 810 for consolidating traffic and minimizing traffic tiransport for 
example. One aspect of tiie present invention concerns aggregating physical 
connections from the customers 830 for presentation to an access router 812. 

Figure 9 iUustrates how tiie LATA IP network 810 can be used to separate 
transport facilities from applications and services. Again, tiie LATA IP network 
810 maybe defined, at least m part, by tiie access routers 812, tiie routers 814, >. 
and tiie interconnection routers 816. Notice tiiat tiie networks of otiiers, such as 
America On-Line, UUNET, SBC, GTE, Sprint and Yahoo may communicate vAih 
tiie LATA IP network 810 via tiie interconnection routers 816. As shown in tiie IP 
application section of Figure 9, the LATA IP network 810 may provide firewall 
functionality (via access router 812), V/IP GW (voice over Internet - gateway), 
next generation switch fimctionaUty (via routers 814), AAA (autiientication, 
autiiorization, and accounting), web caching and video storage facilities (via 
routers 814). The otiier companies may provide chat, e-mail. V/IP GK (voice over 
Internet - gatekeeper) and web hosting functionality via tiieir own networks, and 
the intercormection routers 816. 

The present invention may function to aggregate physical connections 
from customer (also referred to as "client") devices (Recall, e.g., 830 of Figure 8.) 
for presentation to an access router (Recall, e.g., 812 of Figure 8.) and to 
de-aggregate ti^cfromasharedlink(s)fromtiieaccessrouter. (Notetiiata 
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given customer may have multiple devices. Note also that a g^ven customer may 
have more than one service type/level.) The present invention may also function 
to limit or control access to various services thereby performing a firewall 
function. The present invention may also function to enable virtual private 
networks by preserving layer two (2) address information or a imique bit string 
(or context information) in the place of at least some information in the layer 2 
header. The present invention may further function to help provide different 
quality of service levels. Finally, the present invention may fimction to control 
access to multicast groups. 

Figure 10 illustrates connections to, and processes that may be performed 
by, an aggregation imit 1010 of the present invention, as well as processes which 
maybe performed by an access router 812. The aggregation unit 1010 maybe 
coupled with an access router 812 by one or more high bandwidth links 1020- 
Redimdant links 1020 maybe used. Further, links 1050 from a number of 
customers 1030 are coupled with ports 1040 of the aggregation unit 1010. 

The aggregation unit 1010 may perform a port configuration process 1012 
for creating an address table 1060 that maybe used for enabling customer 
addressing, a port aggregation process 1014 which uses information in the 
address table 1060 (See e.g., Figure 29 below.) to manage packets received from 
the ports 1040, a shared link de-aggregation process 1016 which uses information 
in the address table 1060 (See, e.g.. Figure 30 below.) to manage packets received 
from the access router 812, and a multicast group monitoring process 1018 for 
managing access to multicast information using a table 1019 (See, e.g.. Figure 31 
below.). 

Notice that the port configuration process 1012 and the multicast group 
monitor process 1018 may be controlled by, or operate in accordance with, an 
administration entity 1092 which may administer a plan 1090, as indicated by the 
dashed lines. 

The access router 812 may perform an access control process 1082, based 
on an access control list 1083 (See, e.g., Figure 32 below.), a virtual private 
network addressing process 1084 which may use an encapsulatioij lookup table 
1085 (See, e.g., Figure 33 below.), a group service level process 1086, and a group 
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monitor process 1088 for monitoring the service provided to a group of 
customers. These processes may be controlled by, or may operate in accordance 
with, the plan 1090 of the administration entity 1092 as indicated by the dashed 
lines. As shown, a portion of the shared link de-a^egation process 1016' may be 

5 performed by the access router 812 based on a client device address table 1089. 
(See, e-g., Figure 34 below.) 

Having described, at a high level, processes that may be carried out by the 
aggregation unit 1010 and the access router 812, exemplary technologies for 
accessing the aggregation unit 1010 will be described. Then, an exemplary plan 

10 1090, which may be produced and maintained by the administration entity 1092 
will be described. Thereaft'er, an exemplary architecture of the aggregation unit 
1010, as wdl as exemplary data structures of the address table(s) 1060 and other 
aggregation unit table(s) 1019, and exemplary methods for effecting the processes 
of the aggregation unit 1010 will be described. Finally, an exemplary architecture 

15 of the access router 812, as well as exemplary data structures of the access control 
list 1083, an encapsulation lookup table 1085 and an a dient device addressing 
table 1089, and exemplary methods for effecting the processes of the access 
router 812 wiD be described. 

Figure 11 illustrates the manner in which various types of access 
20 technologies may mterface with an access router 812', via an aggregation unit 
1010. To emphasize that the present invention accommodates different access 
technologies, and to illustrate its compatibility with legacy access technologies. 
Figure 11 illustrates how the aggregation units 1010 of the LATA IP network can 
be used with existing (or "legacy") facilities (such as xDSL over ATM 1110 and 
25 native ATM 1140), as well as new access technologies (such as WDM of gigabit 
Ethernet (GbE) 1150). 

In the xDSL over ATM access technology 1110, a customer's computer 1112 
can access an aggregation unit 1010 via an XDSL transmisaon unit-remote at the 
customer premises, wMdi transmits an ATM lo^cal drcuit (or VPI/VCI) 1117 
30 over twisted pair supporting distal subscriber line (or "xDSL") service 1116, to a 
digital subscriber line access multiplexer (or DSLAM") 1130, which connects to a 
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fiber port (for example, OC-3) 1132 of the aggregation irnit, via an ATM logical 
circuit. 

In the ADSL over ATM access technology 1120, a customer's computer 
U22 and Internet telephone 1123 can simultaneously access the aggregation unit 
5 1010 via an ADSL transmission unit-remote ("ATU-R") 1126, over twisted pair 
1116 supporting asymmetrical digital subscriber line (or "ADSL") service, the 
digital subscriber line access multiplexer (or "DSLAM") 1130 and the fiber port 
1132. 

In the ATM access technology 1140, a customer's router 1142 can access 
10 the aggregation unit 1010 via an ATM logical circuit 1144 that connects to a hig^ 
bandwidth port (for example, a 44.736 Mbps DS3 digital line) 1146 on the 
aggregation unit 1010. 

As noted above, the present inventors believe that using DSLAMs with 
ATM ports is not the best or most cost-effective access technology. More 
15 specifically, the present inventors have recognized that IP routed Ethernet can 
offer greater bandwidth, faster failover, simpler operations, better scalability, and 
lower cost than ATM/SONET. Further, IP routed Ethernet may provide 
redimdant management, bus and power. 

Having introduced the ways in which legacy access facilities can interface 
20 with an aggregation unit 1010, an example of how an aggregation vinit 1010' of the 
: present invention may be used to permit new access facilities (such as WDM of 
gigabit Ethernet 1150) is now described. In the example in Figure 11, a customer's 
computer 1152 may interface with the LATA IP network via an optical network 
interface device (or "NID") 1154, over 10/100 Base optical fiber 1156, to a pedestal 
25 (for splidng cables) 1158, that connects to a remote a wave division multiplexer 
(or "WDM") 1160, which connects to a gigabit Ethernet (or "GBE") port 1020* of 
the aggregation unit 1010 

Notice that Ethemet lANs are employed. This is due to their perceived 
cost and performance advantages over other access technologies (such as those 
30 jiist listed above). Although Ethemet is knovm to those skilled in the art, it will 
be described briefly below for the reader's convenience. 



wo 02/191 




PCT/USOl/24732 



- 21- 

Ethemet is a well-known and widely deployed local area network (or 
"LAN") protocol. Ethernet has a bus (as opposed to a ring or star) topology. 
Devices on an Ethernet LAN can transmit whenever they want to — if two (2) or 
more packets collide, each device waits a random time and tries again. More 

5 specifically, as defined in EEEE 802.3, Ethernet is a LAN with persistent carrier 
sense multiple access (or "CSMA") and collision detection (or "CD"). If a device 
wants to transmit, it "listens" to the cable (hence the term "carrier sense"). If the 
cable is sensed as being busy, the device waits of the cable to become idle. If the 
cable is idle, any connected device can transmit (hence the term "multiple 
10 access"). If two (2) or more devices begin to transmit simultaneously, there will 
be a collision which will be detected (hence the term "collision detection"). In the 
event of a collision, the devices causing the collision wiU (i) terminate their 
transmission, (ii) wait a random time, and (iii) try to transmit again (assuming 
that the cable is idle). Accordingly, a CSMA/CD cable or bus has one (1) of three 

15 (3) possible states — contention (or collision), transmission, or idle. Ethernet 

LAN interfaces, like some other IAN interfaces, may have a "promiscuous mode" 
under which all frames are provided to a device, rather than just those addressed 
to the device. 

The IEEE 802.3 frame structure 1200 (or MAC Sublayer Protocol) is 
20 illustrated in Figure 12. The source and destination addresses 1230 and 1240, 
respectively, maybe six (6) bytes (or 48 bits) long. The second most significant 
bit is used to distinguish local addresses from global addresses. Thus, 46 bits are 
available for addresses (or about 7 x 10^3 imique addresses). Accordingly, any 
device can uniquely address any other device by using the right 48-bit address — 
25 it is up to the network layer to figure out how to locate the device associated with 
the destination address. The 48-bit address wHl be discussed in greater detail 
below. 

The two (2) byte length of data field 1250 indicates the number of bytes 
(between o and 1500) present in the data field 1260. At the end of the frame is 
30 the four (4) byte checksum field 1280 that can be used to detect errors in the 
firame. Between the data field and the checksum field is a pad field 1270 of 
variable length. This pad field 1270 is provided because valid fi'aines 1200 must 
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be at least 64 hytes long* Thus, if the data field 1260 is less than 46 bytes, the pad 
field 1270 is used to make both it and the data field 1260 at least 46 bytes. 

Recall that IEEE 802.3 may use frames 1200 which may include 48-bit 
addresses. These addresses may be referred to as media access control (or 
5 **MAC*') addresses. Basically, each device that maybe connected to a network or 
the Internet has an assigned unique MAC address. (Some bits of the MAC 
address are assigned to various device manufactures. The manufactures then 
ensure that each device manufactured by it has a imique MAC address.) 

Although using Ethernet as an access technology to the LATA IP network 
10 introduced above is desirable from a cost and performance standpoint, there are 
certain challenges, met by the present invention, to using this access technology. 
More specifically, unlike legacy access technologies such as asynchronous 
transfer mode (or "ATM") which use end-to-end connections, the Internet 
protocol does not — it is only concerned with the next hop. This presents a 
15 challenge to the owner or operator of the LATA IP network because it cannot 

control the layer 2 (or MAC) and layer 3 (or IP) addresses. For example, because 
the MAC address is assigned to a hardware device such as a NIC, if the ci:istomer 
changes their NIC, their MAC address will change. If the customer adds another 
computer and a router, the MAC address will change to that of the router. 

20 Regarding control by the owner or operator of the LATA IP network of the 

IP address, such an owner or operator may provide service to an Internet service 
provider (or "ISP") for example. Such ISPs typically reserve a number of IP 
addresses that are shared by all of their customers. In this way, the ISP can have 
more customers than reserved addresses. More specifically, the dynamic host 

25 control protocol (or "DHCP") permits the ISP to assign a temporary IP address 
(also referred to as a "dynamic address") to a subscriber. Even the option of 
providing each of an ISP's customers with its own static IP address would become 
unmanageable since every time the ISP added, deleted, or changed the IP address 
of a customer, the LATA IP network owner and/or operator would have to 

30 reconfigure the network. 

In view of the foregoing, the present invention should function to 
aggregate a number* of physical connections to one or more high bandwidth links 
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to an access router. Preferably, the present invention should facQitate the 
deployment of Ethernet access technology. In this regard, the present mvention 
should (i) maintain the identity of the customer device, and (u) maintain address 
information for communicatioiis between the customer device and the access 
5 router 812'. This may be done in accordance with an administered plan, such as 
the one described below. The aggregation unit 1010 of the present invention may 
accomplish these goals by identifying a phyacal or logical port to a customer and 
enablii^ the addressing of the port. Thus, in the present invention, the layer 2 
(e.g., MAC) address is only vmique within the segment to the access router 812'. 
to The present invention may use a plan for forwarding a customer's IP traffic 

that (i) maintains the identity of the source of the packet (e.g., a customer), (ii) 
maintains information regarding where the traffic of the customer device enters 
and exits the LATA IP network, (iii) accommodates all layer 2 access 
technologies, and (iv) permits the provisionmg of service levels to be controlled. 
15 An exemplary plan that may be used to accomplish these goals is described below. 
A plan 109 o*, which may be prepared by an administration entity 1092, 
may identify a logical port of the aggregation unit 1010 to each distinct logical 
drcuit of traffic from a customer device. In this way, each logical port maybe 
configured with enough information to identify the customer that it supports, and 
20 to identifytiiat port in context of all otiier logical ports in tiie IP LATAnetwork. 

An exemplary set of information for such a logical port may include the 
physical interface to which the logical port is attached, the corresponding logic^ 
circuit information for tiie particular access technology, a unique identifier within 
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The 32-bit logical port identifier (or address) may comprise 16 bits that 
define one of 65,536 geographic locations, 4 bits that identify one of sixteen (16) 
physical xmits to which the logical port is attached, and 12 bits that assign one of 
4096 cardinal nimibers to the logical port within its physical unit Naturally, the 
bits of the logical port identifier maybe provisioned based on ingress points, or 
expected future ingress points, to the network 

The present invention may convey the ctistomer addressing information 
among network elements of the LATA IP network using a customer addressing 
protocol that wholly encapsulates the customer's original IP traffic. 

The customer addressing protocol may obtain information from the logical 
port corresponding to a customer's logical circuit. 

The customer addressing protocol maybe in a form of an existing layer 2 
(e.g-, MAC) address or some other unique bits (or context information) in the 
place of, or in addition to the layer 2 address. 

Figure 13 shows an exemplary data structure 1310 for conveying a 
elastomer's identifying information 1312 and customer device addressing 
information 1314- In an exemplary protocol, the data structure 1310 is part of a 
modified Ethernet frame, specifically 88 bits of the 96 bits of addressing space of 
the header. The exemplary protocol replaces the addressing information with a 
24-bit field for the VPN-OUI, a 32-bit field for the VPN-Index, and a 32-bit field 
for the logical port on which the trafl[ic entered the network (or "logical ingress 
port"). This is illustrated in Figure 36. By conveying this information within a 
modified Ethernet firame, the aggregation unit and access router can use any data 
communications technology that supports Ethernet encapsulation of an IP 
packet. That is, the footprint of the Ethernet fi:*ame is not changed. 

This information, in its complete or partial form, may remain attached to 
the original IP packet throughout the LATA IP network. 

Finally, since the information 1310 does not depend on the cont^ts of a 
received packet(s), but rather only on the logical port, this part 1310 of the 
context information can be thought of as a packet-independent part. 
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Tliepresentinventioninayprovideforvariouslevelsof service. In the 

example disclosed, two kinds of service levels are provided: i) quality of servxce; 
andu)dassofservice.Qualityofservice(or"QoS")definesthenetwork 
requirements necessaiytosatisfycertainperformance requirements assoaated 
5 withanIPapplication,forexamplevoiceoverIP.QuaHtyofservicemaybe 

derived from layer 3 and/or 4 information in a received packet(s) and can 
thereforebethot^ht of asapacket-dependent part of the context infomianon. 

aass of service (or "CoS") defines the priority that a customer's IP traffic has 
withinanetwork. Class of service levels maybe customer-selected and can be 
o thoughtofasaservicebundleorsendcelevelagreementCwhichmaybeordered 
and,optiona]ly,modifiedbythecustomer). Since class of service does not 
depend on information in a received packet(s), it canbe thought of as a 
paaet-independentpart of the context information. 

The group service level process 1086 mayrequire service level information 
15 (in addition to the customer device addressing and customer service agreemeiit 
information). Ilxe service level plan maybe prepared by an administration entity 
X092, mayidentifyapackefsQoSby the nature ofits IP application (RecaU 
packetslayer3and/or4iirformation.),andmayidentifyti.esamepacketsCoSby 

reference to additional customer information (e.g.. associated witii the logical 

20 port). 

Given that there is a finite set of popular IP applications, and that a 
taxonomical classification of these applicationsyieldsafinite set of apphcation 

types, an exemplary set of QoS levels mayindude 256 levels, each of which 
correspondstoatypeoflPapplication. Upon receipt of customer ti:affic,tiie 
25 aggregationunitmaydetermmeanS-bitQoStypebyexamimi^tiielayers 

protocol field and/or tiie layer 4 port field. 

Since Cos maybe customer-selected, it maybe part of the customer 
informationset associated withalogical port. TlieCoSforalogicalporti^yuse 

an 8-bit or 16-bit designation, whidi may serve 256 or 65,536 possible CoS levels, 
30 respectively. 

•me present inventioii may convey the service level information among 
network elements of the LATA IP network by extending the context information 
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including the customer identifying and customer device addressing information 
to further convey service level information which may include, or be derived 
from, quality of service and/or class of service information. 

Figure 13 shows an exemplary data structure for conveying service level 
5 ; information 1320 as an extension to a customer identifying and customer device 
addressing part 1310 of the context information. In this exemplary embodiment, 
the context information is extended to include an 8-bit QoS field and an 8-bit or 
16-bit Cos field. The 8-bit (supporting 256 levels) QoS field fits into the 
remaining unused bits (88+8=96) of the 96-bit Ethernet addressing space. The 
10 8-bit or 16-bit class of service (CoS) information may be placed into the Tag ID 
field of an 802.1Q VLAN tag, attached to the Ethernet frame. (See, e.g.. Figure 
• 36.) Alternatively, if an 8-bit CoS is used, the CoS information may be placed into 
the LLC SSAP (link la3^ control - subsystem service access point) field of the 
Ethernet header. 

15 As with the basic context information including customer identifying 

information 1312 and customer device addressing information 1314, the context 
information as extended to include service level information 1320 may remain 
. attached to the original IP packet throughout the LATA IP network. 

In the following, an exemplary architecture of the aggregation unit 1010' is 
20 described with reference to Figures 14 through 18. Then, an exemplary data 

structure for the address table 1060 is described with reference to Figures 29 and 
30. Thereafter, exemplary methods for effecting the processes of the aggregation 
unit are described v\dth reference to Figures 20 through 24. 

Figure 14 is a high-level block diagram which illustrates connections to an 
25 exemplary aggregation unit 1010'. On the right side of the aggregation unit 1010', 
100 10 Mbps fidl duplex ports 1040' per 1 Gbe port or 10 100 Mbps full duplex 
ports per Gbe port may be provided for lines 1050'. On the left side of the 
aggregation unit 1010', a gigabit Ethernet (or "GBE'O link 1020' is provided to the 
access router (not shown). The aggregation unit 1010' may use time division 
30 multiplexing, space division multiplexing (or channelizing), statistical 

.. multiplexing, or another type of multiplexing to aggregate traffic firom the lines 
1050' to the link(s) 1020'. The aggregation unit 1010' maybe a line speed. 
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non-blockmg, unit. In this case, assuming sufficient bandwidth on the link(s) 
1020', 12,000 half-duplex (or 6,000 full-duplex) 10 Mbps customers or 1,200 
half-duplex (or 600 full-duplex) 100 Mbps customers could be accommodated by 
a 1J20 GBE access roijter. Alternatively, the aggregation unit 1010' may 
5 concentrate traffic By providing access facilities capable of providing bandwidth 
that should meet the demands of most foreseeable applications, the present 
invention will allow service levels provided to the customer to be changed without 
changii^ the access fadlities. Thus, for example, a customer could request 
changes in available bandwidth in real time (e.g., via a web interface) tiiat change 
10 the configuration of the logical port (Recall, e.g., plan part 1312 and/or 1320 of 
Figure 13.) to which the customer is connected. 

Figure 15 illustrates an exemplary chassis implementation for an 
aggregation imit 1010'. Network facing interfaces 1520 terminate the high 
bandwidth link(s) 1020' to the access router. Management cards 1510 may be 
15 provided for storing information associated with the ports 1040' (e.g., the logical 
interfaces associated with each port). As will be described below, this 
information maybe assigned during an initial configuration and/or during 
ongoing polling operations. A first management card 1510a mirrors a second 
iSiob. In this way, if one management 1510 card fails, it can be removed, a new 
20 card can be installed, and information can be copied to the newly installed card, 
thereby simplifying maintenance and eliminating any downtime. To the left of 
the management cards 1510 are ports 1040' for terminating lines firom the 
customers. 

In each case, the ports 1040' and network interfaces 1520 have no initial 
25 configiiration. Upon startup or installation, they query the active management 
card 1510 for configviration based on their location in the chassis. Thus, for 
example, a logical interface can be assigned to ports based on their location 
within the LATA IP network (Recall plan part 1314 of Figure 13.), rather than 
solely based on the physical interface card. The bits assigned may be within a 
30 range of bits (or one or more bits of the context information) associated with 
services with which the customer wants. (Recall administration plan 1090* of 
Figure 13.) As discussed above with reference to Figure 14, in on^^ exemplary 
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embodiment, the ports 1040 maybe 10 or 100 Mbps cards, while the network 
. interfaces 1520 maybe iGbps cards. 

Figure 16 is an exemplary management card 1510'. The management card 
includes a data plane 1620, a management plane 1630, flash memory 1610, 
5 indicators 1640 and 1650, such as visual indicators like LEDs for example, and 
management interfaces 1660. 

Figure 17 is an exemplary customer interface card 1700 which includes a 
data plane 1710, a management plane 1720, and a nimiber of hot swappable 
customer ports 1040". Similarly, Figure 18 is an exemplary network interface 
10 card 1800 which includes a data plane 1810, a management plane 1820, and a 
number of hot swappable network interface ports 1520'. 

Basically, processor(s), application specific integrated circuit(s), 
programmable logic array(s), and/or other hardware and/or software maybe 
used to effect the processes of the aggregation unit. 

15 Figures 29 and 30 illustrate exemplary address tables 1060' and 1060", 

respectively, which maybe generated, maintained, and used by the aggregation 
unit 1010. More specifically, these tables 1060' and 1060" maybe configured by 
the port configuration process 1012. The table of Figure 29 maybe used by the 
port aggregation process 1014, and the table of Figure 30 may be used by the 

20 shared link de-aggregation process 1016. 

As shown in Figure 29, the table 1060' may include a column 2910 of 
logical interface or port numbers, a column 2920 of virtual private network 
identifier organizational universal identifiers CVPN-OUI), a column 2930 of 
virtual private network identifier indexes (VPN-Index), a column 2940 of 

25 customer layer 3 (e.g., IP) addresses, a colurrm 2950 of class of service levels, a 
column 2960 of multicast access control list (ACL) groups, a column 2970 of 
quality of service (QoS) profiles, a colimm 2982 of virtual path identifiers (VPIs), 
J a colimm 2984 of virtual channel identifiers (VCIs), a column 2986 of permanent 
virtual drcudts (PVCs), and a coliman 2988 of Ethernet ports. The logical port 

30 nimiber 2910 may be associated with a physical interface 1040' location on the 

chassis. (Recall plan part 1314 of Figure 13.) The VPN-OUI 2920.and VPN-Index 
2930 are also assigned to the port Gogical interface) 1040* by the management 
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card 1510. This asagmnent may be done during initial configuration of the 
aggregation unit loio*. Referrii^ to both Figure 13 and Figure 29, notice that: the 
VPN-OUI column 2920 may correspond to 24 bits of the context information; the 
VPN-Index column 2930 may correspond to 32 bits of the context information; 

5 the VPI 2982, VCI 2984, PVC 2986, and/or ethemet port 2988 columns may 
correspond to other bits of the context information; and the service level 2950, 
multicast access control list group 2960, and/or quality of service profile 2970 
col umns may correspond to other various bits of the context information. To 
reiterate, the table 1060' of Figure 29 maybe used by the port aggregation 

10 process 1014 to aggregate packets from a number of logical interfaces or ports 
onto a link to the access router 812. 

As shown in Figure 30, the table 1060" may include a column 3010 of 
logical interfaces (each of which may correspond to a physical port), a column 
3020 of layer 2 (e.g., MAC) addresses assigned to each of the network-side 
15 interfaces or ports of the aggregation unit, a column 3030 of IP addresses with 
which one or more client device may be associated, a column 3040 of subnet 
masks which maybe used to mask out non-relevant portions of a layer 3 (e.g., IP) 
addi«ss, and a column 3050 of client device layer 2 (e.g., MAC) addresses. A 
layer 3 (e.g., IP) address of column 3030 and a client device layer 2 (e.g., MAC) 
20 address of a client of colimm 3050 may have a one-to-one or one-to-many 

relationship. For example, if a single device, such as a customer computer or a 
company router is always connected to the port, then its IP address and its static 
associated layer 2 (e.g., MAC) address wiU be provided in columns 3030 and 
3050. If, on the other hand, a customer is assigned a dynamic IP address (by its 
25 Internet service provider (or "ISP") and that customer is connected with the port 
through its ISP, for example), then the IP address of column 3050 may have the 
layer 2 (e.g., MAC) address of a customer currently associated with that IP 
address (of the ISP*s router for example). The information in these columns 
3030 and 3050 may be populated by information returned in response to address 
30 resolution broadcasts (e.g., ARPs), and/or by information gleaned by examining 
inbound packet(s) (or "snooping"). The address table 1060" may be used by the 
shared link de-aggregation process ioi6 for example, to forward a packet to the 
proper logical interface or port and to replace the packet's layer 2 (e.g., MAC) 
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destination address (or other information in the place of the layer 2 destination 
address) with that of the customer currently associated with the layer 3 (e.g., IP) 
address. 

In the following, an exemplary method that maybe used to effect the 
5 logical port or interface configuration process 1012 is described with reference to 
Figiires 13 and 20. An exemplary method that maybe used to effect the logical 
port or interface aggregation process 1014 is described with reference to Figures 
21 and 29. An exemplary method that maybe used to effect the shared link 
de-aggregation process 1016 is described with reference to Figures 22 and 30. 
10 Finally, an exemplary method that may be used to effect the multicast group 
monitoring process 1018 is described with reference to Figures 23 and 31. 
■ Generally speaking, processor(s), application specific integrated circuit(s), 
prograrrmiable logic array(s), and/ or other hardware and/or software may be 
used to effect the processes of the access router, 

15 Figure 20 is a flow diagram of an exemplary method 1012* which may be 

used to effect the port configuration process 1012. As shown in optional step 
2010, customers are coupled with ports. More specifically, lines, such as fiber 
. optic lines or copper lines for example, carryrug customer traffic are terminated 
at the ports 1040 of the aggregation imit A logical port is associated with a 

20 physical port or a physical port location as shown in block 2020. (Recall plan 

part 1314 of Figure 13O Customer identifying information and logical ingress port 
information (Recall parts 1312 and 1314 of Figure 13.) maybe provided, as a 
imique bit string (or context information), to the logical port, as shown in step 
2030. Further, class of service information (Recall part 1320 of Figure 13-) may 
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shcmainstep.060. (Recall oolumr^ 3030 and 3050 of Figure 30.) TT^emeUxod 
1012' isleft via MHTURN node 2080 and may be executed as logical ports are , 
added. AttMapoint.thecolunm.otthetablesiaustratedinFigures.9and30 

should be populated. 

Note that the method 1012- can detennine the phydcal port location and 
umque bit string (Recall steps 2020, 2030) at one time, for «.ample upon startup 
of the aggregation unit or when a new customer is added to the ag^t>on umt 
However, the determinaUon of the layer 2 addresses of the atUched dev.ce(s) 
then associated with the layer 3 addresses should tate place periodicaDy. In one 
alten^Uve, aU of the ports periodically poll attached device(s) for its layer 2 
add^ss. Ms poUing should occur frequenUy enough so when the accessrouter 

Sxa' asks it (using for example, an address resoMon) for these addresses, they 

are up to date- 

Figure 21 is a flow diagram of an exemplary method lOM' that may be used 
to effect the port aggregation pnxess tOM' in response to a packet(s) reccved 
ftomacustomer andentering the network, instep 2iio.packet-dependent 

context information (Recall. e.g., QoS of Kgure 13.) is determined based on (e.g.. 
layer3and/orlayer4informationof)thepacket(s)received. Instep2.20 
i^onnationintheori^layer2(e.g.,MACaddresses)headerof.hepacket>s 

removed and the context information is added. The context informaUon may 
include the part asdgned to the lopcal port or interface (packet-independent 
part) and the part determined in step 2U0 (paAet-dependent part). (See. e.g.,. 
Kgure 36.) For example, the layer 2 (e.g.. MAO address assigned to the 
customer device (as well as the layer 2 (e.g.. MAC) address ass«ned to the port) 
■ maybereplacedwithaumquebitstring(orcontextinformation)(e.g.. 

cor^spondmg to the values in columns 2920. 2930. 2950 and 2960 of Bgure 29) 
associated with the lopcal port or intetfece number (See, e.g.. column 2910 of 
Hgure 29.) associated ™th the physicalport 1040 towMchthecustcmeris 
connected, as well as values (e.g.. in «,lumns 2970. 2982. ^984. 2986 and 2988 
ofHgure29)derivedfromUyer3and/orlayer4infonnationinthepacket(s). 

Then, in step 2130. traffic on aU of the lo^cal ports or interfaces is aggre^ted on 
,0 lopcal chamieU on a high bandwidth pferdcal link tt> an accessiouter 812 . 
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This aggregation maybe done via miiltiplexing, such as space division 
mtiltiplexing (channelizing via, e.g., jfirequency division multiplexing, wavelengtii 
division multiplexing, etc.), time division multiplexing, or statistical multiplexing 
for example- As discussed above, in one exemplary embodiment, this aggregation 
5 may be done at line speed, without concentration. The method 1014' is then left 
via RETURN node 2140. To reiterate. Figure 36 illustrates an example of how an 
incoming packet maybe modified by this process 1014. 

Figure 22 is a flow diagram of an exemplary method 1016' which maybe 
used to effect the shared link de-aggregation process 1016 which maybe executed 

10 in response to a packet being received from the network (destined for a 

customer). If a packet has been received from the network, in step 2220, the 
packet is placed on the logical port or interface (See, e.g., column 3010 of Figure 
30.) associated with the information in the layer 2 header of the packet (Recall, 
e.g., part 1314 of Figure 13O Then, in step 2230, the destination layer 2 (e.g., 

15 MAC) address of the packet is changed to that of the customer device associated 
with the logical port or interface. More specifically, referring to Figure 30, the 
layer 2 (e.g., MAC) address of the network side port in coliman 3020 will be 
replaced with the layer 2 (e.g., MAC) address of the customer device in column 
3050 based on the logical port 3010 (and IP address 3030)- The method 1016' is 

20 then left via RETURN node 2240. 

Figure 23 is a flow diagram of an exemplary method 1018* that may be 
. used to effect the multicast group monitoring process 1018. Although 

multicasting using TCP/IP is known to those skilled in the art, it is introduced 
here for the reader's convenience- 

25 Recall from Figure 6A that version 4 of the internet protocol header 

includes 32-bit source and destination addresses. Figure 3 illustrates IP 
compliant addresses. Every host and router on the Internet has a unique IP 
address. Network nimbers are assigned by the Network Information Center (or 
"NIC") to avoid conflicting addresses. This address includes a network number 
' 30 and a host niraiber. Currenfly, there are four (4) classes of address formats. 
Class A permits up to 126 networks with up to 16 million hosts each. Class B 
permits up to 16,382 networks with up to 64,000 hosts each- Class C permits up 
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to 2 million networks with up to 254 hosts each. Class D pennits multicasting. 
Unlike IP address dasses A, B and C, multicasting addresses are not assigned and 
cannot be reserved or controlled by the owner and/or operator of the LATA IP 
network These addresses are controlled by routers v^iich route multicast packets 

5 in accordance with the Internet group multicast protocol (or "IGMP'O- Thus, the 
owner and/or operator of the LATA DP network cannot prevent outaders fix»m 
joining a multicast group by provisioning or controlling multicast addresses. To 
secure the multicast groups, the LATA IP network owner and/or operator may 
manage the multicast address space so that some are reserved for spedfic groups 

10 of customers. In this way, the aggregation unit loio* can deny requests to join a 
multicast group. 

More specifically, referring to step 2310, the method 1018' may examme, 
the bits of the xmique bit string (or context information) that are relevant to 
multicasting. (Recall, e.g., plan parts 1312 class of service 1320 of Figure 13.) If it 

15 is determined that the bit(s) indicate a permission (for a customer) to join a 

particular multicast group, the aggregation unit will provide the packet to the port 
(corresponding to the customer) as shown in steps 2320 and 2330. Otherwise, if 
it is determined that the bit(s) do not indicate a permission for a customer to join 
the particular multicast group, the aggregation imit will block the packet from the 

20 port corresponding to the customer. Although not shown, the packet may be 
forwarded to a port which forwards packets related to network security to a 
monitoring and/or storage facility. The method 1018' is then left via RETURN 
node 2340. 

Figure 31 is a table which illustrates how multicast networks and/or 
25 sub-networks can be assodated with a virtual private network ("VPN"). More 
specificaUy, at least some bits of the VPN-OUI 3140 and VPN-Index 3150 (i.e., 
those bits not masked out by subnet mask 3130) can be assodated with a 
multicast access control list group 3U0 having assodated multicast address 3120. 

Recall from Figure 10 that the access router may perform an access control 
30 process 1082 based on an access control list 1083. A data structure of an 

exemplary access control list is described below with reference to Figures 25 and 
32. Then, an exemplary method that may be used to effect the access control 



WO 02/19056 ^^^■|a^/US01/24732 

-34- 

process is described with reference to Figures 26 and 32, Further recall from 
Figiare 10 that the access router may also perform a virtual private network 
addressing process 1084, a group quality of service process 1086 and a group 
monitor process 1088. An exemplary method that maybe used to effect the 
5 virtual private network addressing process 1084 is described below with reference 
to Figures 27 and 33. An exemplary method that maybe used to effect the group 
service level process 1086 is described below with reference to Figure 28. Finally, 
an exemplary method that may be used to effect the group monitor process 1088 
is described below with reference to Figure 24. Generally speaking, processor(s), 
10 application specific integrated circuit(s), progranmiable logic array(s), and/or 
other hardware and/or software maybe used to effect the processes of the access 
, router. 

Recall from the description of Figure 13 above that a common plan 1090' 
maybe used such that various values of at least some bits of the context 

15 information correspond to various services or customer service agreements. 
(Recall parts 1312 and class of service 1320 of Figure 13.) Figure 25 illustrates a 
data structure of an exemplary access control list 1083' which may be used by the 
access router 812 to permit or deny access to services, locations, etc. More 
; specifically, the Hst 1083' includes a colimm 2510 which lists various values of at 

20 least some bits of the context information (Recall, e.g., Figxire 13.) which 

correspond to various services or customer service agreements. As shown, these 
services may include various services offered by the owner and/or operator of the 
LATA IP network, such as virtual private networks with or without Internet 
access, Internet access only, etc. This information may correspond to the VPN- 

25 OUI 3225, VPN-Index 3230, protocol 3235, L4 port 3240, type of service 3245 
and service level 3250 colimms of Figure 32. Ranges of the layer 3 (e.g., IP) 
source addresses are depicted in the column 2520 (See source IP address 3205 
and mask 3210 columns of Figure 32.), and ranges of the layer 3 (e.g., IP) 
destination addresses are depicted in the coliunn 2530 (See destination IP 

30 address 3215 and mask 3220 columns of Figure 32.). Based on the service bit(s) 
in column 2510, the layer 3 source address and/or the layer 3 destination address, 
the access router 812 can permit or deny a packet, as indicated by coliamn 2540. 
The access router 812 may use these permit/deny instructions to decide whether 
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to route or drop a packet. As canbe appreciated, in this way, various values of 
bitCs) of the context infonnation (as weU as the layer 3 source and/or destination 
address) maybe used to permit or deny access to various services. The last 
instruction in the access control list may be a deny command (if the packet v^as 
not already permitted). An exemplary method that may be carried out the access 

router is described belov^. 

Figure 26 is aflowdiagramof an exemplary method 1082' which may be 
used to effect an access control process 1082. First, as shown in step 2610, any :, 
bit(s) of the context information and/or any bit(s) of layer 2, 3, and/or 4 
addresses that are relevant to access control are examined. (These bits maybe 
taken from the packet using filtering (e.g., masking), etc.) In decision branch 
point 2620, it is determined whether or not thebit(s) indicate a permisaon to 
access a network, a network location, or a sendee for example. (RecaH 
permit/deny column 2540 of Figure 25.) If thebit(s) do mdicate permission to 
access, the packet is routed as shown in step 2630, and the method 1082' is left', 
via RETURN node 2640. Otherwise, the packet is not routed, and the method 
1082' is left via RETURN node 2640. Although not shown, any packets not 
routed maybe forwarded to a port which forwards packets to a network security 
monitorii^ and/or storage facility. 

Figures 27A and 27B are flow diagrams of exemplary methods 1084a' and 
io84b' which may be used to effect a part of the virtual private network 
addressing process 1084. However, the need for these methods will be 
introduced first. 

Recall from Figure 3 that different classes (e.g., A, B, or C) of IP addresses 
can have a different maximum number (e.g., 126, 16,382 or 2,000,000) of 
networks. Although not shown in Figure 3, some of these addresses are not 
uniquely assigned, are not routed by most standard internet routers, and can be 
used by anyone. Thus, more than one company may be uang the same private IP 
address. 

The owner and/or operator of the IP LATA network may want to provide 
virtual private network services. However, as just described, private IP addresses 
arenotnecessarilygloballyunique. The access router 812 may solve this problem 
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as follows. Referring to Figure 27A, at step 2710, at least a portion of an inbound 
packet (e,g., at least a part of the context information) maybe used to identify 
members of a virtual private network. (Recall Figures 29 and 31 and part 1312 of 
Figure 13.) Thus, for example, a company could access the LATA IP network 
5 from more than one access router 812* via more than one aggregation unit 1010*. 
However, each of the ports of the aggregation unit 1010' with which the company 
was connected would include context information having one or more bits which 
could serve to uniquely identify that company's virtual private network. (RecaU, 
plan part 1312 of Figure 13.) This step need only be done once. (Recall step 2030 

10 of the port configuration method illustrated in Figure 20.) At decision branch 
node 2720, it is determined whether or not a packet is received from a customer 
(to be forwarded to the network). If so, a new layer 3 address encapsulates the 
packet so that its unique bit string (or context information), from which a layer 2 
(e-g., MAC) address of the client device can be derived (Recall, e.g., the tables of 

15 Figures 29 and 30), is preserved as shown in step 2730. If this encapsulation 
were not done, the layer 2 address would change over each segment of the 
network. Thus, the encapsulation preserves the concept of group identification, 
service levels, etc. over the entire LATA IP network and not just at the edge of the 
network. Figure 33 illustrates an .exemplary encapsulation lookup table 1085'. 

20 Notice that a new layer 3 destination address 3350 can be derived from at least a 
part of the VPN-OUI 3330 and the VPN-Index 3340. This destination address is 
that of the access router (also referred to as an ^'egress access router" associated 
with the client device having the original layer 3 destination address). 

Figure 37 illustrates an encapsulated IP packet 3700 after routing has been 
25 determined. Notice that the layer 3 source address 3710 is that of the ingress 
access router (i.e., the router performing the encapsulation) and can be 
determined from column 3310 of the table 1085' Figure 33. Notice also that the 
layer 3 destination address 3720 is that of the egress access router (i-e., the access 
router associated with the client device having the original layer 3 destination 
30 address 373o)- The foregoing described the exemplary virtual private network 
addressing method 1084a' from the perspective of a packet entering the network. 
Below, a method 1084b' is described from the perspective of a packet leaving the 
network. 
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^Atobe forwarded to a customer). Kso, the access «uter removes *e 
netwoikCtobetorwaroe The orional layer 3 destination address 

rL 34.) to determi.^ anew VPN-Om (See a>lumn 34.0.). VPN-Into (See 
:Cr343o0.and.helayer,(....M.<.addressofthedesUn^^^^^^ 
device(Seecolunm344o0asshownins.eps^oanda77O im^^^^^^ 
o addresstablero8,.doesnotincl«deent.escorrespondmgtom^_^^^^^ 

destination address, an address resolution request (e.g.. an ABP ) may D 
Weasttorequ^tsuchin^ormationassho^insteps^^^a.^^^^^ 
packet maythenbeforwarded to the aggregation devce as shovmmstep 278 

before the method 1084b' is left via BErORN node 2790. 

Note that al,houghnotshown,beforethepacketisforwardedtov«rds the 

" aggre^:l.««e:es3accessroutercanperformaccess.^an« 

nShtof service processesbasedonat leas, some of .henewb.U(e.g.ton^ 

C OUI and WN-Index). In this way, it the destination customer (chent) has 
~^lev.(e*.servicetypeor,uahty)..henser.ces,^^^ 
,0 limitedby the ingress access router (since the source customer (dev.ce) has 
higher level of service) maybetoitedby the egress router. 

.,^28Uaf.owdiagramofane.empla.— ^^^^^ 
u^dtoeffectthegroupquaU,yofserv.ceprocessro86. F^a^* P 

.8to.anybit(s)ofthecon.e.lnforma.i»a^any^^^^^^ 
:^tr— t^eseblt(s)n«ybee^cted^.heco.^— 

3" -=r:r^t;:;r=^r-:i:::cse.e.g., 

column 3250 of Figure 32.) 
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If the bit(s) indicated a particiilar service level, the packet may be 
forwarded to a queue level associated with the level of priority appropriate for 
that service level as shown in steps 2820 and 2830. The method 1086' is then left 
via RETURN node 2850. 

5 The present invention may also allow packets to or from a particular group 

of customers (e.g., customers from the same company, customers purchasing 
particular quality of service guarantees, etc.) to be copied for monitoring. Figure 
24 is a high level flow diagram of an exemplary method 1088* which may be used 
to effect the group monitoring process 1088. As shown in step 2410, the method 

10 1088' may examine the bit(s) of the imique bit string (or context information) 

and/ or layer 2, 3, and/or 4 addresses to define the group of customers (Recall the 
access control list of Figure 25 and part 1312 of Figure 13.) to be monitored. If it 
is determined that the bit(s) indicate that the customer belongs to the group 
being monitored, the aggregation unit will provide a copy of the packet to a 

15 "monitoring" logical port (not shown) as shown in steps 2420 and 2430. 

Otherwise, if it is determined that the bit(s) do not indicate that the customer 
belongs to the group being monitored, the packet is simply processed as usual. 
The method 1088' is then left via RETURN node 2440. Notice that this method 
1088' is transparent from the perspective of the client devices. 

20 Having described exemplary embodiments of data structures which may 

be used by, and methods which maybe performed by both the aggregation imit 
and the access router, an example which illustrates the end-to-end processing of a 
packet in a system employing these exemplary devices is set forth below. , 

An example which illustrates how a packet maybe sent from a customer to 
25 the network (via an aggregation unit 1010' and an (ingress) access router 812*) 
and how a packet is sent frt)m the network to a customs (via an (egress) access 
router 812' and an aggregation unit loioO is described below, with reference to 
Figures 19, 35, 36 and 37. 

A packet maybe provided from a customer, not shown, to an aggregation 
30 device 1010*. Referring to Figure 35, the packet 3500 is received from the 
; customer 1030' within a layer 2 header that includes a layer 2 (e.g„ MAC) 
destination address 3522 and a layer 2 (e.g., MAC) source address 3524, and may 
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include other fields 3525. 3526. 3527. The layer 3 header 3530 includes a 
protocol field 6i8', a port field 3532, alayer 3 source addressfidd 622', alayer ^ 
destination address field 624', and a type of service field 606'. 

Referring to Figure 19, if the packet is not an address resolution protocol 
(or ARP) packet, as shown by decision block 1902, the aggregation unit 1010' 
changes the layer 2 address information 3522 and 3524 of thelayer 2 header 
3520 (and potentially other information of the layer 2 header 3520, such as field 
3526 for example) to the ingress context information (e.g., the unique bit string) 
associated with the logical port or interface (and derived from the received 
packet(s)) as shown in block 1906. (Recall Figure 29 and step 2120 of Figure 21 
and Figure 13.) Figure 36 illustrates the transfonnation of a packet effected by 
step 906. This new packet 3600 is then passed onto the (ingress) access router 
812' as shown in block 1908. 

still referring to Figure 19. at the access router 812'. an access control list 
(Recall, e.g., Figures 25 and Figure 32.) poHcy maybe applied as shown in block 
1910 and the packet maybe allowed or denied based on the access control list 
policy as shown by dedsionblock 1912. Recall from Figure 25 that the access 
control list may use at least a portion of the unique bit string (or context 
information) replacing the layer 2 header information (See, e.g., column 2510 of 
-o Rgure 25 and columns 3225 and 3230 of Figure 32.) and/or at least a portion of 
the layer 3 address information (See, e.g., columns 2520 and 2530 of Figure 25 
and columns 3205, 3210. 3215 and 3220 of Figure 32.). H the packet is denied 
access, it may be forwarded to a security port '•M2- as indicated by block 1914. If. 
on tiie'otiier hand, the packet is allowed, a type of service may be rewritten as a 
25 -service level" based on layer 2, 3, and/or 4 information as shown in block 1916. 
(See, e.g.. column 3245 of Figure 32 and field 3760 of Figure 37.) 

Next, as shown in blocki9i8 and dedsionblock 1920, a rate limiting 
poUcy may be appHed and enforced. (See, e.g., column 3250 of Figure 32.) Ifthe 
customer (dient) device is exceeding tiie rate spedfied in its dass of service level 
30 agreement, the padcet(s) maybe forwarded to a service level agreement port "Mi" 
as shown by blodi: 1922. If, ontiie otiier hand, the customer (dient) device is 
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within the rate specified in its class of service level agreement, the packet may 
then be forwarded to an encapsiilation interface as shown by block 1924. 

Next, as shown by blocks 1926 and 1928, the layer 2 and 3 addresses, as 
well as the service level are read. (See, e.g.. Figure 32.) Then, as shown by block 
5 1930, the packet is encapsulated with layer 3 information and service level bits 
are set. This encapsulation is shown in Figure 37, wherein the layer 3 (e.g., IP) 
source address 3710 is derived from column 3310 of Rgure 33, the layer 3 (e.g., 
IP) destination address 3720 is derived from column 3350 of Figure 33, and the 
service level value 3760 is derived from the class of service and quality of service 

10 values. (See, e.g., column 3245 of Figure 32 and part 1320 of Figure 13.) The 

layer 2 source address 3740 and the layer 2 destination address 3750 may also be 
written as shown in Figure 37. The layer 2 source address 3712 is known and the 
layer 2 destination address 2714 maybe generated from a lookup table in the 
(ingress) access router 812'. The packet may then be forwarded to the 

15 ; network-facing interface of the access router as shown by block 1932. 

The packet(s) may then be forwarded to the network based on its service 
level. (Recall Figure 28 and part 1320 of Figure 13.) For example, there may 
different queues that have different associated priorities. Packets maybe 
provided to a particular queue based on their service level. The packets then go to 
20 the core IP network 1940 based on some queuing or scheduling discipline. 

Having described the way in which an aggregation unit 1010' and an 
(ingress) access router 812' may handle packets from, a customer destined for the 
core IP network 1940, the way in which an (egress) access router 812' and an 
aggregation unit 1010' may handle packets from the core IP network 1940 
25 destined for a customer is now described. 

As shown by block 1952, a packet(s) received from the core IP network 
1940 is forwarded to a de-encapsulation interface where, as shown by block 1954, 
it is de-encapsulated. (Recall, e.g., step 2750 of Figure 27B.) More specifically, 
referring back to Figure 37, the layer 2 transport and IP encapsulation maybe 
30 stripped from the received packet. 

Then (assuming that layer 3 (e.g., IP) addresses are globally unique), the 
layer 2 destination address (e.g., cUent MAC address) is derived as shown in block 
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1956. For example, referring to the client device addressing table of Figure 34, 
given a layer 3 (e.g., IP) destination address 3410, the unique bit string (or 
context information) (e.g.. the VPN-ID 3420 and 3430) and the layer 2 
destination address 3440 can be derived. (If, on the other hand, it is not assumed 
that the IP addresses are globally unique, a routing poUcy based on the layer 2 
and 3 addresses maybe applied.) The packet is then forwarded to a logical , 
interface of the (egress) access router, as shoAvn inblock 1958, where access 
control and rate limiting policies may be applied based on the new umque bit 
string (or context mformation) (associated with the destination client device 
rather than the source cUent device) as shown in steps i960, 1962, 1964, 1966, 
1968, and 1970. More specifically, at the (egress) access router 812', an access 
control list (Recall, e.g., Figure 25.) policy maybe applied as shown inblock i960, 
and the packet maybe allowed or deniedbased on the access control list pohcy as 
sho^vn by decision block 1962. Recall from Figure 25 that the access control hst 
may use a portion of the unique bit string (or context information) replaong the 
layer 2 address information (See, e.g., column 2510 of Figure 25 and columns 
3225 and 3230 of Figure 32.) and/or a portion of the layer 3 address mformation 
(See, e.g., columns 2520 and 2530 of Figure 25 and columns 3205, 32io, 3215 
and 3220 of Figure 32.) If the packet is denied access, it maybe forwarded to a 
security port ''M2" as indicated by block 1964. If, on the other hand, the packet is 
allowed, as shown in block 1966 and decision block 1968, a rate limiting policy 
maybe applied and enforced. (See, e.g., column 3250 of Figure 32.) If the 
customer (client) device is exceeding the rate specified in its service level 
agreement, the packet(s) maybe forwarded to a service level agreement port «Mi» 
as shown by block 1970. If. on the other hand, the customer (cUent) device is 
within the rate specified in its service level agreement, the packet may thenbe 
forwarded to a network fadng interface of the aggregation device 1010' as shown 
by block 1972. 

As shown in blocks 1982 and 1984, the aggregation device 1010' may 
, forwardthepacketbasedonthelayer2(e.g.,MAC)destinationaddress. Recall 

that this address may have been derived firom the cUent de^^ce addressing table of 
Figure 34. This address may be used to detennine a logical port or interface of 
the aggregation unit 1010'. (Recall, e.g., the address table of Figure 30. 
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Thus, the operations of an aggregation unit loio' and an access router 812' 
on network bound and customer bound packets have been described. 

In view of the foregoing, it is clear that the aggregation unit of the present 
invention may advantageously permit access to an IP network with a robust and 
5 economical access technology such as Ethernet Packets from a large niunber of 
physical line connections can be aggregated onto a smaller number of high 
bandwidth links to an access router. Multicast groups are supported. The service 
provided to groups of customers maybe easily copied for monitoring. The layer 2 
(e.g., MAC) addressing scheme used by the present invention may permit the 
10 access router to control access to various services and locations, to facilitate 
virtual private networks, and to support different quality of service levels. 



wo 02/19( 




PCT/USOl/24732 



WHAT IS CLAIMED IS: 

1. A method for preserving layer 2 address information or information replacing 
a layer 2 address of a client device which sonrced a virtual private network packet, 
the method comprising: 

a) determining a new layer 3 destination address based on at least a 
portion of a layer 3 destination address of the virtual private 
network packet; and 

b) encapsulating the virtual private network packet with a layer 3 
source address, the new layer 3 destination address determined, a 
layer 2 source address and a layer 2 destination address. 



2. The method of claim 1 wherein the layer 3 source address corresponds to the 
layer 3 address of an ingress access router. 

3, The method of claim 1 wherein the new layer 3 destination address determined 
corresponds to the layer 3 address of an egress access router. 



4. A method for forwarding a virtual private network packet in which layer 2 
address information or information replacing a layer 2 address of a device has 
been preserved, in which layer 3 destination address information has been 
preserved and which includes a second layer 3 destination address which 
corresponds to an egress access router, the method comprising: 

a) de-encapsidatibcig the virtual private network packet by removing 
the second layer 3 destination address; 

b) determining a new destination layer 2 address based on (i) at 
least a portion of the preserved layer 3 destination address 
information, and (ii) at least a portion of the layer 2 address 
information or the information replacing the layer 2^ address of the 
device; and 
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c) replacing a destination layer 2 address with the new destination 
layer 2 address determined 

5^ A machine readable storage means having stored thereon a packet sourced 
5 from a client device which defined a layer 3 destination address for the packet 
and which includes a layer 2 source address and a layer 3 source address, the 
packet comprising: 

a) a first field for storing data; 

b) a second field for storing the layer 3 destination address defined by the 
10 source device; 

c) a third field for storing a new layer 3 destination address- 

6. The machine readable storage means of claim 5 wherein the new layer 3 
destination address stored in the third field corresponds to a layer 3 address of an 

15 egress access routen 

7. The machine readable storage means of claim 5 wherein the new layer 3 
destination address stored in the third field is based on at least a portion of the 
layer 3 destination address defined by the source device. 

20 ■ 

8. The machine readable storage means of claim 5 wherein the packet further 
comprises: 

d) a fourth field for storing a bit string associated with a port with which 
the client device sourcing the packet is associated 

9. The machine readable storage means of claim 8 wherein the new layer 3 
destination address stored in the third field is based on at least a portion of the 
layer 3 destination address defined by the client device sourcing the packet and at 
least a portion of the bit string stored in the fourth field. 
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lo The machine readable storage means of daim 8 vvhereinleast a portion of the 
unique bit string stored in the fourthfidd represents one or more services for 

vdiich the client device sourcing the padcet is authorized. 

n The madiine readable storage means of daim 8 v^herein least a portion of the 
uniquebit string stored in the fourth fidd represents a multicast group to which 
the dient device sourcing the packet bdongs. 

12 The madiine readable storage means of daim 8 wherein least a portion of the 
unique bit string stored in the fourth field represents a service levd with whidi 
the dient device sourcing the pad^et is subscribed. 

13 The machine readable storage means of daim 8 wherein least a portion of the 
unique bit string stored in the fourth field represents a location of a logical 
ingress port. 

14. The machine readable storage means of daim 8 wherdnleast a portion of the 
uniquebit string stored in thefourthfidd corresponds to a VPN^UI. 

15 The madiine readable storage means of daim 8 wherein least a portion of the 
uniquebit string stored inthefourthfidd corresponds toaVPN-INDEX. 

16 Anapparatusfor routingvirtualprivatenetworkpackets, eachof thepad^ets 
including layer 2 address information or information replacing a layer 2 address 
of a dient device vMch sourced a virtual private network padcet, the apparatus 
comprL^ng: 
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a) a table including a layer 3 destination address of the virtual private 
network packet and an associated layer 3 address of an egress access 
router; 

b) means for determining a new layer 3 destination address based on the 
5 contents of the table; and 

c) means for encapsulating the virtual private network packet with the 
new layer 3 destination address determined. 



17. A machine readable mediimi having stored thereon a data structure, the data 
10 structure having a plurality of records, each of the records comprising: 

a) a first field for storing a layer 3 destination address; and 

b) a second field for storing a layer 3 address of an egress access router 
associated with layer 3 destination address of the first field, 

wherein the egress access router is a router at the edge of a network. 

15 

, 18. The machine readable medium of daim 17, each of the records further 
comprising: 

c)a third field for storing a string of bits in the place of a layer 2 address 
associated with the client device which sourced the virtual private 
20 network packet. 
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